This repository was archived by the owner on Aug 2, 2025. It is now read-only.
Conversation
* Advanced logging * remove docker.io/library from images * updated entrypoint for cup integration * added swagger and more * updated new routes and added first routes which will be controlled by the frontend * Added auth (still buggy here and there) * fixed auth problem when trying to use swagger * Add more logging * added databse functionality * test for auto commit message * test auto git changelog * Update changelog.yml * Update changelog.yml * test new CHANGELOG.md * Changed Files: -------------- .github/workflows/changelog.yml CHANGELOG.md data/database.db package-lock.json package.json * Added offline development capabilities * Advanced frontend customization and new dependecy graph generator using mermaid diagrams and dependecy-cruiser * Dependency cruiser and gitignore adjustmnts * better port assignment, when running in a non-docker environment * Adjust workflows and adjusted entrypoint.sh file * Rate limiter * Rate limiter * adjust workflows --------- Co-authored-by: ItsNik <nniklaskrause@duck.com> Co-authored-by: root <root@DESKTOP-JQC6J06>
…information from /frontend/...
Closed
22 tasks
* Full ES6 support * Full ES6 support * TODO: fix npm run dev in ts * TODO: fix 'ERROR : Error fetching data: ' * Delete files * Adding more typing; making code more logical at some points; * Added typing and fixed > dockstatapi@2 dep > bash ./src/utils/createDependencyGraph.sh Route: frontend Route: auth ./routes/frontendController/routes.ts ./routes/auth/routes.ts Route: data ./routes/data/routes.ts Route: notificationService ./routes/notifications/routes.ts Route: api ./routes/getter/routes.ts Route: conf ./routes/setter/routes.ts ======== DONE ======== * Added typing and fixed 'npm run dep' * Advanced logging and fixing some bugs * Adjust workflows * New README and some other docer adjustments * First time building (god damn) * Fixing some typings * Fixing some typings in default notification modules * Needs fixing! * Needs fixing! * Create CodeQL.yml * Fixing more errors * Added some more typings and better /api/status route * New mermaid deiagrams * New mermaid deiagrams * HA route * No building errors! * Remove CodeQL * Use selfhosted runners * nerver mind, using too much ressources on my cloud machine * Added HA functionality (Please test) * Fixing package versions * Fix: Creating default config if it doesn't exist * Fix: Sync endpoint * Fix: Endpoint reachability check * Chore: Update notification service * CI: Added cloc * Fix: adjust cloc workflox * Fix: exclude node modules from cloc * Fix: we have no yaml files, probably due to some other actions or smth * Fix: exclude package-lock from cloc * Created new dependency graphs * Feat: Added playwright tests for API endpoint + Swagger test (auth) (#23) Co-authored-by: ItsNik <info@itsnik.de> * Chore: Cleanup * Fix: Added proxy support * Chore: Custom notifications * Feat: minified build Chore: Alpine based Dockerfile Chore: Advance Lifecycle scripts included in dockstatapi@2: start tsx src/server.ts available via `npm run-script`: start:build npx tsc && export NODE_NO_WARNINGS=1 && node dist/server.js dev nodemon dev:trace nodemon --trace-uncaught --trace-warnings dep bash ./src/utils/createDependencyGraph.sh dep:remove bash ./src/utils/removeUnusedDeps.sh && bash ./src/utils/createDependencyGraph.sh build npx tsc build:mini npx tsc && bash ./src/misc/minifyDist.sh --build-only mini bash ./src/misc/minifyDist.sh scripts * Feat: minified build Chore: Alpine based Dockerfile Chore: Advance Lifecycle scripts included in dockstatapi@2: start tsx src/server.ts available via `npm run-script`: start:build npx tsc && export NODE_NO_WARNINGS=1 && node dist/server.js dev nodemon dev:trace nodemon --trace-uncaught --trace-warnings dep bash ./src/utils/createDependencyGraph.sh dep:remove bash ./src/utils/removeUnusedDeps.sh && bash ./src/utils/createDependencyGraph.sh build npx tsc build:mini npx tsc && bash ./src/misc/minifyDist.sh --build-only mini bash ./src/misc/minifyDist.sh scripts * Better docker image * Chore: Update docker image to alpine base * Fix: use find instead of tree in minifyDist.sh * Chore: Update Readme * Fix: Force correct node version (took some time to find it) * Fix: Typo in package.json * Feat: added npmc * Fix: Remove data-bak * Feat: Switch to yarn in Dockerfile * Feat: Switch to yarn in Dockerfile * Fix: Yarn does not work => Change to npm * Fix: Specify Node version using nvmrc in workflow file * Test: Try npm i --verbose to see why workflow times out * Test: Try with all environment files * Warn: Removing arm/v7 support due to docker build incompatibilities * Chore: Add test build for debugging with dockstatapi:<Commit SHA> * Chore: Add opencontainer labels * Chore: Added automatic notifications Chore: Add init.ts instead of one big server.ts Fix: Allow usePassword.txt in ./src/data (previously not included) * Fix: Adjusted .gitignore * Chore: Changed from ? true : false to simpler syntax * Chore: Added lock file when a sync is running * Chore: Remove any typing highAvailability.ts Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * Chore: Update src/utils/connectionChecker.ts Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * Fix: Missing Typing for HA * Fix: Environment Varaible usage inside docker * Fix: Forgot the copying of the file inside the Dockerfile (bruh) --------- Co-authored-by: ItsNik <info@itsnik.de> Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Reviewer's Guide by SourceryThis pull request implements DockStatAPI v2, introducing significant backend improvements and new features. Sequence diagram for High Availability synchronizationsequenceDiagram
participant M as Master Node
participant W as Worker Node
participant D as Docker Socket
M->>M: Monitor config changes
M->>W: Sync configuration files
W->>W: Update local config
M->>D: Fetch container stats
W->>D: Fetch container stats
D-->>M: Return stats
D-->>W: Return stats
M->>M: Store in database
W->>W: Store in database
Entity Relationship diagram for DockStatAPI v2 data modelerDiagram
CONTAINER ||--o{ STATS : has
CONTAINER {
string id
string name
string state
string hostName
string networkMode
}
STATS {
float cpu_usage
int mem_usage
int mem_limit
int net_rx
int net_tx
timestamp created_at
}
CONTAINER ||--o{ FRONTEND_CONFIG : has
FRONTEND_CONFIG {
boolean hidden
string[] tags
boolean pinned
string link
string icon
}
Class diagram for Frontend ConfigurationclassDiagram
class FrontendConfig {
+name: string
+hidden: boolean
+tags: string[]
+pinned: boolean
+link: string
+icon: string
}
class ContainerData {
+name: string
+id: string
+hostName: string
+state: string
+cpu_usage: number
+mem_usage: number
+mem_limit: number
+net_rx: number
+net_tx: number
+networkMode: string
}
class HighAvailabilityConfig {
+active: boolean
+master: boolean
+nodes: string[]
}
ContainerData -- FrontendConfig
HighAvailabilityConfig -- ContainerData
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey @Its4Nik - I've reviewed your changes - here's some feedback:
Overall Comments:
- Consider expanding test coverage beyond the basic API tests - add unit tests for core functionality like high availability, container management, and notification services
- The high availability feature needs more detailed documentation, particularly around setup, failover behavior, and operational considerations
Here's what I looked at during the review
- 🟡 General issues: 4 issues found
- 🟡 Security: 2 issues found
- 🟡 Testing: 2 issues found
- 🟡 Complexity: 2 issues found
- 🟢 Documentation: all looks good
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
|
|
||
| const discord_webhook_url: string | undefined = process.env.DISCORD_WEBHOOK_URL; | ||
|
|
||
| export async function discordNotification(containerId: string): Promise<void> { |
There was a problem hiding this comment.
suggestion: Consider extracting common HTTP request logic into a shared utility function
The HTTP request logic is duplicated across Discord, Slack, and Telegram notifications. Creating a shared utility would reduce code duplication and make maintenance easier.
Suggested implementation:
import logger from "../logger";
import { renderTemplate } from "./_template";
import { sendHttpNotification } from "./httpUtils";export async function discordNotification(containerId: string): Promise<void> {
const discord_message: string | null = renderTemplate(containerId);
if (!discord_message) {
logger.error("Failed to create notification message.");
return;
}
if (!discord_webhook_url) {
logger.error("Discord webhook URL is not set.");
return;
}
await sendHttpNotification({
url: discord_webhook_url,
data: {You'll need to create a new file src/utils/notifications/httpUtils.ts with the following content:
import * as https from 'https';
interface NotificationRequest {
url: string;
data: Record<string, any>;
}
export async function sendHttpNotification({ url, data }: NotificationRequest): Promise<void> {
const postData = JSON.stringify(data);
const options = {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Content-Length': Buffer.byteLength(postData),
},
};
return new Promise((resolve, reject) => {
const req = https.request(url, options, (res) => {
let responseData = '';
res.on('data', (chunk) => {
responseData += chunk;
});
res.on('end', () => {
if (res.statusCode && res.statusCode >= 200 && res.statusCode < 300) {
resolve();
} else {
reject(new Error(`HTTP request failed with status ${res.statusCode}`));
}
});
});
req.on('error', (error) => {
reject(error);
});
req.write(postData);
req.end();
});
}You'll also need to update the Slack and Telegram notification functions to use this shared utility function in a similar way.
| logger.info( | ||
| `Creating Docker client for host: ${hostConfig.url} on port: ${hostConfig.port || 2375}`, | ||
| ); | ||
| return new Docker({ |
There was a problem hiding this comment.
🚨 suggestion (security): Add support for TLS/HTTPS connections to Docker daemon
The Docker client should support secure TLS connections. Consider adding ca, cert and key options when protocol is https.
Suggested implementation:
interface DockerHostConfig {
name: string;
url: string;
port?: number;
protocol?: 'http' | 'https';
ca?: string;
cert?: string;
key?: string;
}
function createDockerClient(hostConfig: DockerHostConfig): Docker {
logger.info(
`Creating Docker client for host: ${hostConfig.url} on port: ${hostConfig.port || 2375} using ${hostConfig.protocol || 'http'}`,
);
const options: any = {
host: hostConfig.url,
port: hostConfig.port || 2375,
protocol: hostConfig.protocol || 'http',
};
if (hostConfig.protocol === 'https') {
if (hostConfig.ca) options.ca = fs.readFileSync(hostConfig.ca);
if (hostConfig.cert) options.cert = fs.readFileSync(hostConfig.cert);
if (hostConfig.key) options.key = fs.readFileSync(hostConfig.key);
}
return new Docker(options);
}You'll also need to:
- Import the fs module at the top of the file:
import * as fs from 'fs'; - Update your Docker configuration JSON schema to include the new TLS-related fields
- Update any existing Docker host configurations to specify protocol and include paths to TLS certificate files when using HTTPS
Example docker config.json format:
{
"hosts": [
{
"name": "production",
"url": "tcp://docker-host",
"port": 2376,
"protocol": "https",
"ca": "/path/to/ca.pem",
"cert": "/path/to/cert.pem",
"key": "/path/to/key.pem"
}
]
}
src/utils/containerService.ts
Outdated
| } | ||
| } | ||
|
|
||
| async function fetchAllContainers(): Promise<AllContainerData> { |
There was a problem hiding this comment.
issue (complexity): Consider refactoring the fetchAllContainers function by extracting container processing logic into separate functions.
The fetchAllContainers function could be simplified by extracting the container processing logic. This would reduce nesting and improve maintainability while keeping the same functionality. Consider this refactoring:
async function processContainer(docker: any, container: ContainerInfo, hostName: string): Promise<ContainerData> {
try {
const containerInstance = docker.getContainer(container.Id);
const [containerInfo, containerStats] = await Promise.all([
containerInstance.inspect(),
containerInstance.stats({ stream: false })
]);
const cpuUsage = calculateCpuUsage(containerStats);
return {
name: container.Names[0].replace("/", ""),
id: container.Id,
hostName,
state: container.State,
cpu_usage: cpuUsage * 1000000000,
mem_usage: containerStats.memory_stats.usage,
mem_limit: containerStats.memory_stats.limit,
net_rx: containerStats.networks?.eth0?.rx_bytes || 0,
net_tx: containerStats.networks?.eth0?.tx_bytes || 0,
current_net_rx: containerStats.networks?.eth0?.rx_bytes || 0,
current_net_tx: containerStats.networks?.eth0?.tx_bytes || 0,
networkMode: containerInfo.HostConfig.NetworkMode || "unknown",
};
} catch (error: any) {
logger.error(
`Error fetching details for container ID: ${container.Id} on host: ${hostName} - ${error.message}`,
);
return createEmptyContainerData(container, hostName);
}
}
function calculateCpuUsage(stats: ContainerStats): number {
const cpuDelta = stats.cpu_stats.cpu_usage.total_usage - stats.precpu_stats.cpu_usage.total_usage;
const systemCpuDelta = stats.cpu_stats.system_cpu_usage - stats.precpu_stats.system_cpu_usage;
return systemCpuDelta > 0 ? (cpuDelta / systemCpuDelta) * stats.cpu_stats.online_cpus : 0;
}
function createEmptyContainerData(container: ContainerInfo, hostName: string): ContainerData {
return {
name: container.Names[0].replace("/", ""),
id: container.Id,
hostName,
state: container.State,
cpu_usage: 0,
mem_usage: 0,
mem_limit: 0,
net_rx: 0,
net_tx: 0,
current_net_rx: 0,
current_net_tx: 0,
networkMode: "unknown",
};
}This refactoring:
- Reduces nesting depth in the main function
- Makes the code more testable by isolating container processing logic
- Improves readability by grouping related operations
- Parallelizes container info and stats fetching
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| const ResponseHandler = createResponseHandler(res); | ||
| const host: string = (req.query.host as string) || "local"; | ||
|
|
||
| logger.info(`Fetching containers from host: ${host}`); |
Check warning
Code scanning / CodeQL
Log injection Medium
| throw new Error(error as string); | ||
| } | ||
| } else { | ||
| logger.error(`Provided link is not valid: ${link}`); |
Check warning
Code scanning / CodeQL
Log injection Medium
| try { | ||
| await fs.promises.writeFile( | ||
| dataPath, | ||
| JSON.stringify(data, null, 2), |
Check warning
Code scanning / CodeQL
Network data written to file Medium
| } | ||
|
|
||
| rawData(data: unknown, message: string) { | ||
| logger.info(message); |
Check warning
Code scanning / CodeQL
Log injection Medium
| }); | ||
|
|
||
| req.on("error", (error) => { | ||
| logger.error("Error sending Slack message:", error); |
Check warning
Code scanning / CodeQL
Log injection Medium
| }); | ||
|
|
||
| req.on("error", (error) => { | ||
| logger.error("Error sending message:", error); |
Check warning
Code scanning / CodeQL
Log injection Medium
| logger.error("Error sending message:", error); | ||
| }); | ||
|
|
||
| req.write(postData); |
Check warning
Code scanning / CodeQL
File data in outbound network request Medium
| }); | ||
|
|
||
| req.on("error", (error) => { | ||
| logger.error("Error sending WhatsApp message:", error); |
Check warning
Code scanning / CodeQL
Log injection Medium
| logger.error("Error sending WhatsApp message:", error); | ||
| }); | ||
|
|
||
| req.write(postData); |
Check warning
Code scanning / CodeQL
File data in outbound network request Medium
* Chore: new testing workflows (dropping playwright) * Chore: updating github workflow * Fix: Fixing some minor things * Chore: Updated to ES2020 syntax and AMD module * Feat: startServer function to start the server with a different port * Fix: Adjusting testing files based on workflow restrictions * Fix: Adjusting testing files based on workflow restrictions * Chore: Updating swagger (wrong branch bruh) * Docs: Update swagger documentation (#26) * Chore: Updated swagger * Fix: Typo * Fix: Fixing dockerfiles for prod/dev environment * Feat: Add `/graph` and `/graph/image` endpoints (#27) * Feat: Server side HTML generation => Client side rendering * Fix: This _might_ fix the workflow * Fix: Remove unused function * Fix: Please make it stop * Fix: Setting up python before hand * Fix: Remove unused dep * Fix: Using node20 instead of latest * Fix: Works on my end... * Feat: Master Nodes * Feat: Icon for master node (needs testing) * Fix: Adjusting function (needs testing) * Fix: Adjusting function (needs testing) * Fix: Removed some graph rendering features (will be back but better) * Feat: render html file as png using puppeteer ToFix: svgs dont render * Fix: Hell yeah we got image creation! * Fix: Adjusted routes since they need an absolute path * Fix: Remove unused dependencies * Fix: Exclude CWE-200 from CodeQl * Feat: Respomse examples in swagger Fix: Fixing some catch blocks * Fix: Adjusting catches * Fix: Adjusted catch to typing * Feat: Stack creation * Feat: Stack creation + starting and stopping * Fix: Project root instead of path Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * Fix: Logging adustment * Fix: Allow undescores and dashes in stack name Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * Fix: Propagate error * Fix: Inline variable that is immediately returned Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * Fix: move some things around * Fix: Minor adjustments * Feat: Get a stack's docker-compose * Feat: automatic Stack environmental file management * Fix: sample-varaible.json adjustment * Fix: Potential fix for code scanning alert no. 102: Log injection Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> * Fix: fix for code scanning alert no. 94: Uncontrolled data used in path expression Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> * Fix: fix for code scanning alert no. 92: Uncontrolled data used in path expression Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> * FiX: fix for code scanning alert no. 106: Log injection Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> * Fix: Logger vulnerability and CI graph generation * Feat: change logger verbosity and spelling fix * Feat: ToDo comments to GH issue * Fix: Add checkout * Fix: May fix the ToDo workflow * Fix: Remove todo * Fix: Re-Add commit * Fix: Remove TODO * Fix: Re-add TODO * Fix: Where tf did my package lock go 😭 * CI/CD: Remove ToDo * CI/CD: Add ToDo * CI/CD: Fix command * CI/CD: Add checkout * Fix: CPU value was a percentage the whole time? * Feat: Websocket endpoints for logs and container metrics * Fix: Make linter happy * Fix: Fix import * Fix: Fix tsc build * Jest: Fix tests * Jest: Fix Tests * Fix: Typo in src/config/swagger.yaml Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * Fix: Typo in src/config/swagger.yaml Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * Fix: Tyypo in src/config/swagger.yaml Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * (code-quality): Inline variable that is immediately returned Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * (code-quality): Prefer object destructuring when accessing and using properties. Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * (code-quality): Prefer object destructuring when accessing and using properties. Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * (code-quality): Prefer object destructuring when accessing and using properties. Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * Fix: Update extractHostData.ts * Update TODO.md --------- Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Co-authored-by: ItsNik <info@itsnik.de>
| throw new Error(`File already exists: ${targetPath}`); | ||
| } | ||
|
|
||
| fs.writeFileSync(tempFile, writeData, { mode }); |
Check warning
Code scanning / CodeQL
Network data written to file Medium
|
|
||
| if (!hostConfig) { | ||
| const errorMsg = `Docker host ${hostName} not found in configuration`; | ||
| logger.error(errorMsg); |
Check warning
Code scanning / CodeQL
Log injection Medium
Owner
Author
Please see v3 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Main Goal:
Overall way better backend
For more information follow this Issue: #16
Summary by Sourcery
Introduce DockStatAPI v2 with breaking changes. Update documentation and dependencies.
New Features:
Tests: